Skip to main content

​
Overview

The fetch_advanced_indicators.py script retrieves advanced technical indicators for all stocks, including pivot points, exponential/simple moving averages (EMA/SMA), and technical sentiment signals. This data powers the technical analysis fields in the final output.

​
Purpose

Fetches technical indicator data including:
  • Pivot Points - Classic, Fibonacci, Camarilla pivot levels
  • EMA (Exponential Moving Average) - 20, 50, 100, 200 period EMAs
  • SMA (Simple Moving Average) - 20, 50, 100, 200 period SMAs
  • Technical Indicators - RSI sentiment, MACD signals, Bollinger Band status

​
API Endpoint

​
URL
string
required
https://ow-static-scanx.dhan.co/staticscanx/indicator
​
Method
string
required
POST

​
Request Payload

{
  "exchange": "NSE",
  "segment": "E",
  "security_id": "<Sid>",
  "isin": "<ISIN>",
  "symbol": "<SYMBOL>",
  "minute": "D"
}

​
Parameters

​
exchange
string
default:"NSE"
required
Exchange code (NSE for National Stock Exchange)
​
segment
string
default:"E"
required
Market segment (E for Equity)
​
security_id
string
required
Security ID (Sid) from master_isin_map.json. This is required for this API.
​
isin
string
required
ISIN code of the security
​
symbol
string
required
Stock symbol (e.g., “RELIANCE”)
​
minute
string
default:"D"
required
Timeframe interval:
  • D - Daily (most common)
  • W - Weekly
  • M - Monthly
  • 1, 5, 15, 30, 60 - Intraday minutes

​
Output Files

​
advanced_indicator_data.json
array
Consolidated indicator data for all stocks (~8.3 MB). Each object contains:
{
  "Symbol": "RELIANCE",
  "EMA": [
    {"period": 20, "value": 2450.30, "signal": "Above"},
    {"period": 50, "value": 2380.15, "signal": "Above"},
    {"period": 200, "value": 2200.50, "signal": "Above"}
  ],
  "SMA": [
    {"period": 20, "value": 2455.20, "signal": "Below"},
    {"period": 50, "value": 2385.40, "signal": "Above"}
  ],
  "TechnicalIndicators": [
    {"name": "RSI", "value": 62.5, "signal": "Neutral"},
    {"name": "MACD", "signal": "Bullish"}
  ],
  "Pivots": [
    {"type": "Classic", "pivot": 2450.00, "R1": 2480.00, "S1": 2420.00},
    {"type": "Fibonacci", "pivot": 2450.00, "R1": 2468.00, "S1": 2432.00}
  ]
}

​
Function Signature

def fetch_indicators(item):
    """
    Fetches advanced indicators for a single stock.
    
    Args:
        item (dict): Stock object with 'Symbol', 'ISIN', and 'Sid' keys
        
    Returns:
        dict or None: Indicator data object or None if Sid missing or request fails
        
    Process:
        1. Validate that Sid exists (required for this API)
        2. Construct payload with exchange, segment, security_id, isin, symbol
        3. POST request to indicator API
        4. Extract EMA, SMA, TechnicalIndicators, and Pivots from response
        5. Return structured data
    """

​
Dependencies

​
Python Packages
list
  • requests - HTTP client
  • json - JSON processing
  • os - File operations
  • concurrent.futures.ThreadPoolExecutor - Parallel execution
​
Local Modules
list
  • pipeline_utils.BASE_DIR - Base directory path
  • pipeline_utils.get_headers() - Standard API headers
​
Input Files
list
  • master_isin_map.json - Must include Sid field for each stock

​
Threading Configuration

​
MAX_THREADS
number
default:"50"
Number of concurrent threads for fast parallel execution

​
Code Example

import requests
import json
import os
from concurrent.futures import ThreadPoolExecutor, as_completed
from pipeline_utils import BASE_DIR, get_headers

INPUT_FILE = os.path.join(BASE_DIR, "master_isin_map.json")
OUTPUT_FILE = os.path.join(BASE_DIR, "advanced_indicator_data.json")
MAX_THREADS = 50

def fetch_indicators(item):
    """Fetch advanced indicators for a stock."""
    symbol = item.get("Symbol")
    isin = item.get("ISIN")
    sid = item.get("Sid")
    
    # Sid is required for this API
    if not sid:
        return None

    api_url = "https://ow-static-scanx.dhan.co/staticscanx/indicator"
    headers = get_headers()
    
    payload = {
        "exchange": "NSE",
        "segment": "E",
        "security_id": str(sid),
        "isin": isin,
        "symbol": symbol,
        "minute": "D"  # Daily timeframe
    }

    try:
        response = requests.post(api_url, json=payload, headers=headers, timeout=10)
        if response.status_code == 200:
            data = response.json().get("data", [])
            if data and isinstance(data, list) and len(data) > 0:
                result = data[0]
                
                # Extract specific indicator fields
                ema_data = result.get("EMA", [])
                sma_data = result.get("SMA", [])
                indicators = result.get("Indicator", [])
                pivots = result.get("Pivot", [])
                
                return {
                    "Symbol": symbol,
                    "EMA": ema_data,
                    "SMA": sma_data,
                    "TechnicalIndicators": indicators,
                    "Pivots": pivots
                }
        return None
    except:
        return None

def main():
    if not os.path.exists(INPUT_FILE):
        print(f"Error: {INPUT_FILE} not found. Run fetch_dhan_data.py first.")
        return

    with open(INPUT_FILE, "r") as f:
        master_list = json.load(f)

    print(f"Starting indicator fetch for {len(master_list)} stocks...")
    all_results = []
    
    with ThreadPoolExecutor(max_workers=MAX_THREADS) as executor:
        future_to_stock = {executor.submit(fetch_indicators, stock): stock for stock in master_list}
        
        completed = 0
        for future in as_completed(future_to_stock):
            res = future.result()
            if res:
                all_results.append(res)
            completed += 1
            if completed % 100 == 0:
                print(f"Progress: {completed}/{len(master_list)} done.")

    with open(OUTPUT_FILE, "w") as f:
        json.dump(all_results, f, indent=4)

    print(f"Saved indicators for {len(all_results)} stocks to {OUTPUT_FILE}")

​
Usage

python3 fetch_advanced_indicators.py

​
Performance

  • Execution Time: ~1-2 minutes for 2,775 stocks
  • API Calls: 2,775 requests (one per stock)
  • Output Size: ~8.3 MB
  • Concurrency: 50 parallel threads
  • Success Rate: High (stocks without Sid are skipped)

​
Critical Requirement: Sid Field

This API requires the Sid (Security ID) field. Stocks without a valid Sid will be skipped.Ensure master_isin_map.json contains the Sid field by running fetch_dhan_data.py first.

​
Indicator Types Fetched

​
1. Pivot Points

  • Classic Pivot - Traditional pivot calculation
  • Fibonacci Pivot - Fibonacci-based support/resistance
  • Camarilla Pivot - Intraday pivot levels
  • Each includes: Pivot, R1, R2, R3, S1, S2, S3

​
2. Moving Averages

  • EMA: 20, 50, 100, 200 periods
  • SMA: 20, 50, 100, 200 periods
  • Each includes: period, value, signal (Above/Below price)

​
3. Technical Signals

  • RSI (14) - Relative Strength Index with sentiment (Overbought/Neutral/Oversold)
  • MACD - MACD signal (Bullish/Bearish/Neutral)
  • Bollinger Bands - Band position and volatility metrics

​
Notes

  • Uses daily timeframe (minute: "D") by default
  • 10-second timeout per request
  • Stocks without Sid are silently skipped (not counted as errors)
  • Results are not sorted (maintain insertion order)
  • High concurrency (50 threads) is safe for this endpoint